Anti-Deprecation: Towards Complete Static Checking for API Evolution Extended Version

API evolution is the process of migrating an inter-library interface from one version to another. Such a migration requires checking that all libraries which interact through the interface be updated. Libraries can be updated one by one if there is a transition period during which both updated and non-updated libraries can communicate through some transitional version of the interface. Static type checking can verify that all libraries have been updated, and thus that a transition period may end and the interface be moved forward safely. Anti-deprecation is a novel type-checking feature that allows static checking for more interface evolutions periods. Anti-deprecation, along with the more familiar deprecation, is formally studied as an extension to Featherweight Java. This formal study unearths weaknesses in two widely used deprecation checkers. “In Java when you add a new method to an interface, you break all your clients.... Since changing interfaces breaks clients you should consider them as immutable once you’ve published them.” –Erich Gamma [21] “NoSuchMethodError” –Java VM, all too frequently 1. OVERVIEW Libraries communicate with each other via application programming interfaces (API’s), or interfaces for short. The key idea with interfaces is that so long as a set of libraries conform to their interfaces, those libraries will tend to function together when they are combined. This approach is a key part of standard discussions of software modularity [2]. This interfaces idea supports independent evolution of liThis is an extended version of a workshop paper [19]. LCSD 2006 Portland, Oregon, USA braries, in that libraries can be updated so long as they continue to conform to their interfaces. However, this strategy does not address evolution of the interfaces themselves. Since in practice the first definition of an interface is often insufficient, practitioners need some approach for improving interfaces. This is the problem of interface evolution. Interface evolution arises in practice for large-scale projects with multiple independent development groups. The Eclipse project, for example, includes plugin code written by development groups all over the world. For such projects, substantial attention is put onto the problem of safely upgrading interfaces [5]. Transition periods provide a general mechanism for evolving the interfaces between independently maintained libraries. A transition period is a period of time during which both updated and non-updated libraries can successfully communicate through an evolving interface. During a transition period, all libraries that conform to the original version of an interface must be allowed to continue to function. As the transition period progresses, more and more libraries should be updated for the forthcoming version of the interface, while continuing to work with the transitional version of the interface. A transition period can successfully terminate when all libraries communicating through the interface have been either updated or abandoned. At that time, the interface itself can be upgraded. Static type checking can be used to verify that a transition period may be safely entered or left. At the beginning of a transition period, static checking can ensure that all libraries conforming to the current interface will continue to conform to the new, transitional interface. At the end of a transition period, static checking can ensure that all checked libraries are ready to progress to the next version of the interface. The same checker can be used for both purposes if the checker has two strictness levels. The strict level is used to check the exit from transition periods, while the looser transitional level is used for all other type-checking purposes. This article studies static type checking for deprecation and public interface ConnectionListener { public void connectionClosed(); public void connectionClosedOnError(Exception e); } public interface ConnectionListener2 extends ConnectionListener { public void connectionAuthenticated(); } Figure 1: Two interfaces from Eclipse. The second interface is the same as the first except that it requires one new method. public interface ConnectionListener { public void connectionClosed(); public void connectionClosedOnError(Exception e); encouraged public void connectionAuthenticated(); } Figure 2: With encouraged methods, the new method could have been gradually phased into the original interface. anti-deprecation of methods. Deprecation is widely used, while anti-deprecation appears to be novel for programming languages. After describing the features in general, the article defines them formally as an extension to Featherweight Java [11], and proves several core properties about the formalism. This systematic study not only defines the new feature, but unearths two places where current deprecation checkers could be improved. 2. STATIC TRANSITION CHECKING Static checking can help both entering and leaving transition periods. When entering a transition period, the checker can verify that clients will continue to compile and run, even if not all libraries using the interface are available. As the transition period moves forward, each library’s developers can use the checker as they update their library to verify that their updates are sufficient for the next version of the interface. Once all libraries have been updated and checked, it is safe to move the interface forward. Put another way, the entries and exits of transition periods are refactorings [14]. If the static checker is satisfied, then crossing these end points causes a change in program syntax but not in program behavior. Not all libraries need to be available to those maintaining the interface. The conditions for entering a transition period are typically weak, thus giving interface maintainers broad liberty to start an interface transition. Leaving the transition period requires more work, but it does not need to be finished immediately. Every library whose components use the interface must be checked with the strict checker, but those checks can occur throughout the transition period. Once the (loose) organization of library maintainers have decided that sufficient checking has occurred, and if no errors are known to be present, the transition period can be left. Organization processes for deciding that enough library assemblies have been checked that a transition period may be left are beyond the scope of this article. Presumably, however, some such agreement has been reached among the library developers. As one example arrangement, the maintainers of the interface might commit to a minimum length of evolution period. That length might be e.g. six months, a year, or five years. Anyone building assemblies that use that interface must periodically check their library, with a period no longer than the agreed length of evolution periods. A static transition checker can be described as having two modes: transitional and strict. If a library passes the transitional checker, then the library can communicate with other libraries through the interface. If a library additionally passes the strict checker, then the library will also continue to work if the interface is updated. The strict checker takes into consideration extra annotations describing the desired interface changes, while the transitional checker mostly ignores such annotations. Implementations can combine the two checking modes. All code must pass the transitional checker, while failure to additionally pass the strict checker causes interface-evolution warnings. 3. ANTI-DEPRECATION Deprecation allows a static checker to emit warnings whenever a caller tries to use a method that is expected to disappear in a future version of an interface. A complementary scenario is also important: sometimes a future version of an interface will require an additional method. An annotation for such future required methods could be called anti-deprecation. The typical usage for anti-deprecation is shown in Figures 1 and 2. Figure 1 shows one of Eclipse’s “I*2” interfaces, an interface that is an extension of an earlier interface. Experience with the framework showed that the earlier interface was too thin, but given the nature of Java interfaces, new methods could not be added to the existing, published interface. Thus, the Eclipse developers added a second interface which merely extends the first interface and adds one new method. With encouraged methods, the designers would have had the option to phase in the method to the existing interface, as shown in Figure 2. A simple way to annotate anti-deprecation is to add an encouraged keyword to the language. Unlike other methods, a method marked as encouraged cannot be called. Its presence only serves to mark that a future version of the interface will include that same method as abstract. During transitional checking, encouraged methods are, for the most part, treated as if they were not present at all. The only restriction is that encouraged methods cannot override other non-encouraged methods. Allowing such would be complicated and unhelpful—after all, if a method is already present due to inheritance, what use is it to encourage it further? The one exception, that encouraged methods can nonetheless override other encouraged methods, is necessary L ::= class C extends C { C̄ f̄ ; K X̄ M̄ } X ::= deprecated m; K ::= C(C̄ f̄) { super(f̄); this.f̄ = f̄ ; } M ::= C m(C̄ f̄) MB MB ::= { return e ; } | abstract | encouraged e ::= x | e.f | e.m(ē) | new C(ē) | (C)e Figure 3: Syntax of FJ-ADE so that encouraged methods can be added over other encouraged methods. In strict checking, even this case is not allowed, and the encouraged method deeper in the hierarchy needs to be removed. During strict checking, encouraged methods add several requirements for programs to pass the checker. First, any method that overrides an encouraged method must have the required parameter types and return type. This requirement is present so that when an encouraged method is later promoted to a required method, all methods overriding it will have conforming types. Second, every subclass of a class with an encouraged method must either implement the method or be considered abstract and uninstantiable. The combination of deprecation and anti-deprecation allows for an additional class of changes that neither mechanism supports alone: arbitrary changes to a method’s signature. For example, one might wish to change the set of exceptions thrown by a method, or change a method’s return type, or change its public or private visibility. Such changes can always be accomplished using four transition periods. The first period introduces a new version of the method with a different name than the original method. Since the method is new, it can be given any type signature at all. The second period deprecates the original method, thus inducing callers to use the new version of the method. The third period replaces the deprecated original method with an encouraged method of the desired signature. The fourth period deprecates the temporary method name, thus inducing clients to change back to using the original method. Alternatively, developers can choose a shorter two-phase sequence if they are content for the new method to have a different name from the original. They can simply stop after the first two transition phases. These rules for encouraged and deprecated might seem pessimistic. These rules are formed under the assumption that developers in other groups might both implement any interface and invoke the methods it advertises. If this assumption were changed to restrict what other developers can do, then some interface changes could be safely performed with fewer or even no transition periods. For example, suppose that one party controls an interface along with all of its implementors. In that case, that party can add methods to the interface without needing a transition period. They can simply make a simultaneous release of the updated interface and the updated implementors of that interface. Likewise, if one party controls all callers to an interface, e.g. as with call backs, that party can remove methods from the interface without needing a transition period. The present work addresses the less constrained scenario where outside developers can both implement an interface and call through it. The main reason for this choice is that it is the more general and difficult case. However, notice that even when outside developers are expected to be more constrained in their work, it is desirable to allow them the greater flexibility. At the least, it is useful for testing if programmers can implement their own mock objects to stand in place of the usual ones [12, 9]. 4. EXTENDING FEATHERWEIGHT JAVA While deprecated and encouraged are simple to describe, it proves tricky to develop the precise rules for checking them so that transition periods can be safely entered and left. In order to determine the precise checking rules, the bulk of this article focuses on a formal study of a small language including these keywords. The keywords are added to Featherweight Java (FJ) [11], a language that has several appealing characteristics: it is tiny, making it amenable to formal study; it uses familiar syntax, so that the work is more approachable; and it captures two features at the heart of object-oriented languages, message sending and inheritance. In one way, though, the FJ language is a little too small for the present purpose: it does not include a notion of interfaces. Instead of adding a full interface concept, it suffices to add abstract methods. Abstract methods allow abstract classes, which for the present purpose serve as perfectly fine interfaces. The full extended language is called FJ-ADE because it is Featherweight Java with three new keywords: abstract, deprecated, and encouraged. The notation is generally that of FJ. When a line of code is written down by itself as an assumption, the meaning is that that line of code appears somewhere in the program. A sequence is written x̄, denoting the sequence x1, . . . , xn, where #(x̄) = n. The empty sequence is • by itself, while a comma between two sequences denotes concatenation. Pairs of sequences are a shorthand for a sequence of pairs; for example, C̄ x̄ means C1 x1 . . . Cn xn. The notation x ∈ ȳ means that x = yi for some i. Negation, written ¬P , is not boolean negation, but instead means that P cannot be proven with the available inference rules. The syntax of FJ-ADE is given in Figure 3. There are a few differences from FJ: • Methods can be abstract. Any class that defines or inherits an abstract method is considered abstract and cannot be instantiated with new. • Methods can be encouraged. An encouraged method will be added to a future version of the class with the specified type signature. • Each class has a list of deprecated methods. Deprecated methods are going to be removed in a future T-Var x : C ∈ Γ Γ ` x : C T-Field Γ ` e0 : C0 fields(C0) = C̄ f̄ Γ ` e0.fi : Ci T-New fields(C) = D̄ f̄ str ; Γ ` ē : C̄ C̄ <: D̄ ¬abstract(C) (str = trans) ∨ (¬postabs(C)) str ; Γ ` new C(ē) : C T-Invk str ; Γ ` e0 : C0 mtype(m, C0, false, str = trans) = D̄ → C str ; Γ ` ē : C̄ C̄ <: D̄ str ; Γ ` e0.m(ē) : C T-UCast Γ ` e0 : D D <: C Γ ` (C)e0 : C T-DCast Γ ` e0 : D C <: D C 6= D Γ ` (C)e0 : C T-SCast Γ ` e0 : D D 6<: C D 6<: C stupid warning Γ ` (C)e0 : C Figure 4: Typing of expressions T-Method-Fresh str ; x̄ : C̄, this : C ` e0 : E0 E0 <: C0 class C extends D {. . . } ¬mavail(m, D, (str = strict), true) C0 m(C̄ x̄) { return e0; } str−OK IN C T-Method-Over str ; x̄ : C̄, this : C ` e0 : E0 E0 <: C0 class C extends D {. . . } mtype(m, D, (str = strict), true) = D̄ → D0 C̄ = D̄ C0 = D0 C0 m(C̄ x̄) { return e0; } str−OK IN C T-Method-Abs class C extends D {. . . } ¬mavail(m, D, (str = strict), true) C0 m(C̄ x̄) abstract str−OK IN C T-Method-Enc class C extends D {. . . } ¬mavail(m, D, (str = strict), true) C0 m(C̄ x̄) encouraged str−OK IN C Figure 5: Typing of methods T-Class K = C(D̄ ḡ, C̄ f̄) { super(ḡ); this.f̄ = f̄ ; } fields(D) = D̄ ḡ M̄ str−OK IN C ∀m ∈ X̄ : candep(C, m) class C extends D { C̄ f̄ ; K X̄ M̄ } str−OK Figure 6: Typing of classes